Kemp shared successive stands of 65 with Mark Boucher (33) for the sixth wicket and Shaun Pollock (37) for the seventh as South Africa pulled off a victory that stretched their unbeaten streak to 15 one-day internationals.
There was a wobble towards the end when Pollock and Kemp were both caught on the boundary within the space of three balls in the 49th over against off-spinner Jeetan Patel when only three runs were needed.
But Nicky Boje hit a two off the first ball of the final over from Shane Bond before, with all the fielders up to prevent a single, he skied the third delivery just out of the reach of a diving New Zealand captain, Stephen Fleming, for the winning run.
Both Kemp and Pollock survived chances before taking South Africa to their target of 250.
Kemp hit a return catch to Nathan Astle when he was on eight and made the medium-pacer pay dearly in his next over when he took 22 off Astle, including two fours and two successive sixes.
Pollock was also dropped on eight when Patel at deep midwicket missed a chance off Kyle Mills.
Fleming said New Zealand's 249 for eight was a "reasonable total" on a slow pitch against accurate bowling and lively fielding.
Craig McMillan top-scored with 66 and shared a fifth wicket stand of 75 off 103 balls with Jacob Oram (35).
New Zealand made a horror start in the field when Shane Bond sent down two wides in the first over, both of which went for four extra runs. The over cost 17 runs.
After two overs South Africa were 25 for no wicket.
But South Africa slumped to 41 for three and fell well behind the required run rate. The normally aggressive Herschelle Gibbs, making a comeback after a knee injury, took 69 balls to score 25.
Boucher and Kemp gave South Africa some hope before Pollock joined Kemp in a partnership which put South Africa on top.
Kemp hit his 73 off 64 balls, with three sixes and five fours, while Pollock made 37 off 29 balls with a six and two fours.
SCOREBOARD
=======================================
NEW ZEALAND
---------------------------------------
S. Fleming c Gibbs b Nel 45
N. Astle c Boucher b Pollock 1
L. Vincent c Boucher b Ntini 8
H. Marshall lbw b Kallis 16
C. McMillan c Prince b Kallis 66
J. Oram run out (Smith) 35
B. McCullum c Dippenaar b Nel 28
D. Vettori c Prince b Nel 1
A. Adams not out 30
K. Mills not out 1
EXTRAS: (B-1, LB-11, NB-2, W-4) 18
---------------------------------------
TOTAL: (for 8 wkts, 50 overs) 249
=======================================
FALL OF WICKETS: 1-7, 2-18, 3-70, 4-97, 5-172, 6-200, 7-204, 8-241
BOWLING: Pollock 10-2-43-1, Ntini 4-0-19-1 (2nb), Langeveldt 10-1-48-0 (1w), Nel 9-0-42-3, Kallis 9-1-45-2 (2w), Boje 6-0-32-0 (1w), Kemp 2-0-8-0.
DID NOT BAT: S Bond.
SUBSTITUTE (not used): J Patel.
=======================================
South Africa
---------------------------------------
G Smith b Adams 19
H Dippenaar lbw b Bond 10
J Kallis b Mills 0
H Gibbs c Marshall b Bond 25
A Prince b Astle 31
M Boucher b Vettori 33
J Kemp c Vincent b Patel 73
S Pollock c Adams b Patel 37
N Boje not out 3
C Langeveldt not out 0
Extras: (b1, lb4, nb1, w13) 19
---------------------------------------
Total: (8 wkts, 49.3 overs) 250
=======================================
Fall of wickets: 1-29, 2-34, 3-41, 4-85, 5-117, 6-182, 7-247, 8-247.
Bowling: Bond 9.3-1-50-2 (10w), Mills 10-2-38-1, Adams 6-0-28-1, Vettori 10-0-35-1 (1nb), Astle 6-0-46-1 (1w), Patel 8-0-48-2 (2w).
UMPIRES: S Bucknor (WI), I Howell (RSA).
TV UMPIRE: K Hurter (RSA).
MATCH REFEREE: C Broad (ENG).
Result: South Africa won by two wickets.
Man-of-the-match: Justin Kemp.